Adding detailed telemetry#561
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive telemetry tracking to monitor usage of different DQX methods and storage operations. It introduces a telemetry framework to capture method calls for analytics and monitoring purposes.
- Added a new telemetry module with decorator-based logging functionality
- Instrumented engine methods and storage handlers with telemetry decorators
- Updated documentation to use "in-transit" terminology instead of "in-flight"
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/databricks/labs/dqx/telemetry.py | New telemetry module with logging functions and decorator |
| src/databricks/labs/dqx/engine.py | Added telemetry decorators to engine methods and individual check logging |
| src/databricks/labs/dqx/checks_storage.py | Added telemetry decorators to all storage handler methods |
| pyproject.toml | Added "ui/*" to ruff exclude list |
| docs/dqx/docs/reference/engine.mdx | Minor documentation fixes (trailing spaces) |
| docs/dqx/docs/guide/index.mdx | Changed "in-flight" to "in-transit" terminology |
| demos/dqx_demo_pii_detection.py | Changed "in-flight" to "in-transit" terminology |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
✅ 295/295 passed, 4 flaky, 1 skipped, 3h1m33s total Flaky tests:
Running from acceptance #2293 |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
alexott
left a comment
alexott
left a comment
There was a problem hiding this comment.
small comments, otherwise looks good to me
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 3 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Changes
Linked issues
Resolves #378
Tests